Intention-Sensing Recipe Guidance via User Accessing to Objects

Sensing the intention of a user’s forthcoming action is a necessary function for systems that assist human physical activity. In this paper, we investigate a strategy for recipe guidance systems that can predict the forthcoming intended sub-task in a cooking task. We focus on user accessing objects, i.e., touching and releasing objects. Touching can indicate the start of the forthcoming sub-task and releasing can indicate the end of the task. The main difficulty lies in the fact that humans may move objects because they are in the way and use cooking tools that are unanticipated by an assistive system. In such cases, the accessed object should not indicate the forthcoming sub-task. Our contribution is that we propose a method to track the progress of a task based on the object access history. This enables us to eliminate object accesses that are out of context. Simultaneously, the method predicts the forthcoming sub-task based on a combination of progress and materials rather than tools and materials. We develop a guidance system that runs as a web service. In experiments, we observe real cooking activities navigated by this system. The Wizard of OZ method is utilized to simulate a system that detects object accesses. The experimental results show that we achieve 73.6% accuracy in the selection of the displayed information. This result supports the use of “access to objects” realize effective intention-sensing systems.